Transformer-based language models have shown an excellent ability to effectively capture and utilize contextual information. Although various analysis techniques have been used to quantify and trace the contribution of single contextual cues to a target task such as subject-verb agreement or coreference resolution, scenarios in which multiple relevant cues are available in the context remain underexplored. In this paper, we investigate how language models handle gender agreement when multiple gender cue words are present, each capable of independently disambiguating a target gender pronoun. We analyze two widely used Transformer-based models: BERT, an encoder-based, and GPT-2, a decoder-based model. Our analysis employs two complementary approaches: context mixing analysis, which tracks information flow within the model, and a variant of activation patching, which measures the impact of cues on the model's prediction. We find that BERT tends to prioritize the first cue in the context to form both the target word representations and the model's prediction, while GPT-2 relies more on the final cue. Our findings reveal striking differences in how encoder-based and decoder-based models prioritize and use contextual information for their predictions.

It can't dribble, let alone slam dunk, but Toyota's basketball robot hardly ever misses a free throw or a 3-pointer. The 207-centimeter-tall (6 feet 10-inches) machine made five of eight 3-point shots in a demonstration in a Tokyo suburb Monday, a ratio its engineers say is worse than usual. Toyota Motor Corp.'s robot, called Cue 3, computes a three-dimensional image where the basket is, using sensors on its torso, and adjusts motors inside its arm and knees to give the shot the right angle and propulsion for a swish. Efforts in developing human-shaped robots underline a global shift in robotics use from pre-programmed mechanical arms in limited situations like factories to functioning in the real world with people. The 2017 version of the robot was designed to make free throws.